Eye Tracking in the Wild: the Good, the Bad and the Ugly

Modelling human cognition and behaviour in rich naturalistic settings and under conditions of free movement of the head and body is a major goal of visual science. Eye tracking has turned out to be an excellent physiological means to investigate how we visually interact with complex 3D environments, real and virtual. This review begins with a philosophical look at the advantages (the Good) and the disadvantages (the Bad) in approaches with different levels of ecological naturalness (traditional tightly controlled laboratory tasks, low- and high-fidelity simulators, fully naturalistic real-world studies). We then discuss in more technical terms the differences in approach required “in the wild”, compared to “received” lab-based methods. We highlight how the unreflecting application of lab-based analysis methods, terminology, and tacit assumptions can lead to poor experimental design or even spurious results (the Ugly). The aim is not to present a “cookbook” of best practices, but to raise awareness of some of the special concerns that naturalistic research brings about. References to helpful literature are provided along the way. The aim is to provide an overview of the landscape from the point of view of a researcher planning serious basic research on the human mind and behaviou

Gaze Strategies in Driving : An Ecological Approach

Human performance in natural environments is deeply impressive, and still much beyond current AI. Experimental techniques, such as eye tracking, may be useful to understand the cognitive basis of this performance, and “the human advantage.” Driving is domain where these techniques may deployed, in tasks ranging from rigorously controlled laboratory settings through high-fidelity simulations to naturalistic experiments in the wild. This research has revealed robust patterns that can be reliably identified and replicated in the field and reproduced in the lab. The purpose of this review is to cover the basics of what is known about these gaze behaviors, and some of their implications for understanding visually guided steering. The phenomena reviewed will be of interest to those working on any domain where visual guidance and control with similar task demands is involved (e.g., many sports). The paper is intended to be accessible to the non-specialist, without oversimplifying the complexity of real-world visual behavior. The literature reviewed will provide an information base useful for researchers working on oculomotor behaviors and physiology in the lab who wish to extend their research into more naturalistic locomotor tasks, or researchers in more applied fields (sports, transportation) who wish to bring aspects of the real-world ecology under experimental scrutiny. Part of a Research Topic on Gaze Strategies in Closed Self-paced tasks, this aspect of the driving task is discussed. It is in particular emphasized why it is important to carefully separate the visual strategies driving (quite closed and self-paced) from visual behaviors relevant to other forms of driver behavior (an open-ended menagerie of behaviors). There is always a balance to strike between ecological complexity and experimental control. One way to reconcile these demands is to look for natural, real-world tasks and behavior that are rich enough to be interesting yet sufficiently constrained and well-understood to be replicated in simulators and the lab. This ecological approach to driving as a model behavior and the way the connection between “lab” and “real world” can be spanned in this research is of interest to anyone keen to develop more ecologically representative designs for studying human gaze behavior.Peer reviewe

Eye movements in the wild : Oculomotor control, gaze behavior & frames of reference

Understanding the brain's capacity to encode complex visual information from a scene and to transform it into a coherent perception of 3D space and into well-coordinated motor commands are among the outstanding questions in the study of integrative brain function. Eye movement methodologies have allowed us to begin addressing these questions in increasingly naturalistic tasks, where eye and body movements are ubiquitous and, therefore, the applicability of most traditional neuroscience methods restricted. This review explores foundational issues in (1) how oculomotor and motor control in lab experiments extrapolates into more complex settings and (2) how real-world gaze behavior in turn decomposes into more elementary eye movement patterns. We review the received typology of oculomotor patterns in laboratory tasks, and how they map onto naturalistic gaze behavior (or not). We discuss the multiple coordinate systems needed to represent visual gaze strategies, how the choice of reference frame affects the description of eye movements, and the related but conceptually distinct issue of coordinate transformations between internal representations within the brain.Peer reviewe

Egocentric Chunking in the Predictive Brain : A Cognitive Basis of Expert Performance in High-Speed Sports

Publisher Copyright: Copyright © 2022 Lappi. First publication by Frontiers Media.What principles and mechanisms allow humans to encode complex 3D information, and how can it be so fast, so accurately and so flexibly transformed into coordinated action? How do these processes work when developed to the limit of human physiological and cognitive capacity—as they are in high-speed sports, such as alpine skiing or motor racing? High-speed sports present not only physical challenges, but present some of the biggest perceptual-cognitive demands for the brain. The skill of these elite athletes is in many ways an attractive model for studying human performance “in the wild”, and its neurocognitive basis. This article presents a framework theory for how these abilities may be realized in high-speed sports. It draws on a careful analysis of the case of the motorsport athlete, as well as theoretical concepts from: (1) cognitive neuroscience of wayfinding, steering, and driving; (2) cognitive psychology of expertise; (3) cognitive modeling and machine learning; (4) human-in-the loop modellling in vehicle system dynamics and human performance engineering; (5) experimental research (in the laboratory and in the field) on human visual guidance. The distinctive contribution is the way these are integrated, and the concept of chunking is used in a novel way to analyze a high-speed sport. The mechanisms invoked are domain-general, and not specific to motorsport or the use of a particular type of vehicle (or any vehicle for that matter); the egocentric chunking hypothesis should therefore apply to any dynamic task that requires similar core skills. It offers a framework for neuroscientists, psychologists, engineers, and computer scientists working in the field of expert sports performance, and may be useful in translating fundamental research into theory-based insight and recommendations for improving real-world elite performance. Specific experimental predictions and applicability of the hypotheses to other sports are discussed.Peer reviewe

The Racer's Mind—How Core Perceptual-Cognitive Expertise Is Reflected in Deliberate Practice Procedures in Professional Motorsport

The exceptional performance of elite practitioners in domains like sports or chess is not a reflection of just exceptional general cognitive ability or innate sensorimotor superiority. Decades of research on expert performance has consistently shown that experts in all fields go to extraordinary lengths to acquire their perceptual-cognitive and motor abilities. Deliberate Practice (DP) refers to special (sub)tasks that are designed to give immediate and accurate feedback and performed repetitively with the explicit goal of improving performance. DP is generally agreed to be one of the key ingredients in acquisition of expertise (not necessarily the only one). Analyzing in detail the specific aspects of performance targeted by DP procedures may shed light on the underlying cognitive processes that support expert performance. Document analysis of professional coaching literature is one knowledge elicitation method that can be used in the early phases of inquiry to glean domain information about the skills experts in a field are required to develop. In this study this approach is applied to the domain of motor racing - specifically the perceptual-cognitive expertise enabling high-speed curve negotiation. A systematic review procedure is used to establish a corpus of texts covering the entire 60 years of professional motorsport textbooks. Descriptions of specific training procedures (that can be unambiguously interpreted as DP procedures) are extracted, and then analyzed within the hierarchical task analysis framework driver modeling. Hypotheses about the underlying cognitive processes are developed on the basis of this material. In the traditional psychological literature, steering and longitudinal control are typically considered “simple” reactive tracking tasks (model-free feedback control). The present findings suggest that—as in other forms expertise—expert level driving skill is in fact dependent on vast body of knowledge, and driven by top-down information. The knowledge elicitation in this study represents a first step toward a deeper psychological understanding of the complex cognitive underpinnings of expert performance in this domain

Visuomotor control, eye movements, and steering : A unified approach for incorporating feedback, feedforward, and internal models

The authors present an approach to the coordination of eye movements and locomotion in naturalistic steering tasks. It is based on recent empirical research, in particular, on driver eye movements, that poses challenges for existing accounts of how we visually steer a course. They first analyze how the ideas of feedback and feedforward processes and internal models are treated in control theoretical steering models within vision science and engineering, which share an underlying architecture but have historically developed in very separate ways. The authors then show how these traditions can be naturally (re)integrated with each other and with contemporary neuroscience, to better understand the skill and gaze strategies involved. They then propose a conceptual model that (a) gives a unified account to the coordination of gaze and steering control, (b) incorporates higher-level path planning, and (c) draws on the literature on paired forward and inverse models in predictive control. Although each of these (a–c) has been considered before (also in the context of driving), integrating them into a single framework and the authors’ multiple waypoint identification hypothesis within that framework are novel. The proposed hypothesis is relevant to all forms of visually guided locomotion.Peer reviewe

Eye-movements in real curve driving: pursuit-like optokinesis in vehicle frame of reference, stability in an allocentric reference coordinate system

Looking at the future path and/or the tangent point (TP) have been identified as car drivers’ gaze targets in many studies on curve driving. Yet little is known in detail about these "fixations to the road". We quantitatively analyse gaze behavior at the level of individual fixations in real on-road data. We find that while gaze tracks the TP area, this pattern consists of fast optokinetic movements (smooth pursuit and fast resetting saccadic movements). Gaze is not “fixed” to the TP. We also relate eye-movements to a reference direction fixed to a point on the trajectory of the vehicle (curve exit), showing that fixations lose their pursuit-like character in this rotating system. The findings are discussed in terms of steering models and neural levels of oculomotor control

Inattention and Uncertainty in the Predictive Brain

Negative effects of inattention on task performance can be seen in many contexts of society and human behavior, such as traffic, work, and sports. In traffic, inattention is one of the most frequently cited causal factors in accidents. In order to identify inattention and mitigate its negative effects, there is a need for quantifying attentional demands of dynamic tasks, with a credible basis in cognitive modeling and neuroscience. Recent developments in cognitive science have led to theories of cognition suggesting that brains are an advanced prediction engine. The function of this prediction engine is to support perception and action by continuously matching incoming sensory input with top-down predictions of the input, generated by hierarchical models of the statistical regularities and causal relationships in the world. Based on the capacity of this predictive processing framework to explain various mental phenomena and neural data, we suggest it also provides a plausible theoretical and neural basis for modeling attentional demand and attentional capacity “in the wild” in terms of uncertainty and prediction error. We outline a predictive processing approach to the study of attentional demand and inattention in driving, based on neurologically-inspired theories of uncertainty processing and experimental research combining brain imaging, visual occlusion and computational modeling. A proper understanding of uncertainty processing would enable comparison of driver's uncertainty to a normative level of appropriate uncertainty, and thereby improve definition and detection of inattentive driving. This is the necessary first step toward applications such as attention monitoring systems for conventional and semi-automated driving.Peer reviewe

Anticipatory eye movements when approaching a curve on a rural road depend on working memory load

Peer reviewe

Systematic Observation of an Expert Driver's Gaze Strategy—An On-Road Case Study

In this paper we present and qualitatively analyze an expert driver’s gaze behavior in natural driving on a real road, with no specific experimental task or instruction. Previous eye tracking research on naturalistic tasks has revealed recurring patterns of gaze behavior that are surprisingly regular and repeatable. Lappi (2016) identified in the literature seven “qualitative laws of gaze behavior in the wild”: recurring patterns that tend to go together, the more so the more naturalistic the setting, all of them expected in extended sequences of fully naturalistic behavior. However, no study to date has observed all in a single experiment. Here, we wanted to do just that: present observations supporting all the “laws” in a single behavioral sequence by a single subject. We discuss the laws in terms of unresolved issues in driver modeling and open challenges for experimental and theoretical development.Peer reviewe